{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Keras\n",
    "\n",
    "Keras 是一个用于构建和训练深度学习模型的高阶 API。它可用于快速设计原型、高级研究和生产。\n",
    "\n",
    "keras的3个优点：\n",
    "方便用户使用、模块化和可组合、易于扩展\n",
    "\n",
    "\n",
    "##  1.导入tf.keras\n",
    "tensorflow2推荐使用keras构建网络，常见的神经网络都包含在keras.layer中(最新的tf.keras的版本可能和keras不同)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.0.0-alpha0\n",
      "2.2.4-tf\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow.keras import layers\n",
    "print(tf.__version__)\n",
    "print(tf.keras.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.构建简单模型\n",
    "### 2.1模型堆叠\n",
    "最常见的模型类型是层的堆叠：tf.keras.Sequential 模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = tf.keras.Sequential()\n",
    "model.add(layers.Dense(32, activation='relu'))\n",
    "model.add(layers.Dense(32, activation='relu'))\n",
    "model.add(layers.Dense(10, activation='softmax'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2网络配置\n",
    "\n",
    "tf.keras.layers中网络配置：\n",
    "\n",
    "activation：设置层的激活函数。此参数由内置函数的名称指定，或指定为可调用对象。默认情况下，系统不会应用任何激活函数。\n",
    "\n",
    "kernel_initializer 和 bias_initializer：创建层权重（核和偏差）的初始化方案。此参数是一个名称或可调用对象，默认为 \"Glorot uniform\" 初始化器。\n",
    "\n",
    "kernel_regularizer 和 bias_regularizer：应用层权重（核和偏差）的正则化方案，例如 L1 或 L2 正则化。默认情况下，系统不会应用正则化函数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.layers.core.Dense at 0x7f99c444ddd8>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "layers.Dense(32, activation='sigmoid')\n",
    "layers.Dense(32, activation=tf.sigmoid)\n",
    "layers.Dense(32, kernel_initializer='orthogonal')\n",
    "layers.Dense(32, kernel_initializer=tf.keras.initializers.glorot_normal)\n",
    "layers.Dense(32, kernel_regularizer=tf.keras.regularizers.l2(0.01))\n",
    "layers.Dense(32, kernel_regularizer=tf.keras.regularizers.l1(0.01))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.训练和评估\n",
    "### 3.1设置训练流程\n",
    "构建好模型后，通过调用 compile 方法配置该模型的学习流程：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = tf.keras.Sequential()\n",
    "model.add(layers.Dense(32, activation='relu'))\n",
    "model.add(layers.Dense(32, activation='relu'))\n",
    "model.add(layers.Dense(10, activation='softmax'))\n",
    "model.compile(optimizer=tf.keras.optimizers.Adam(0.001),\n",
    "             loss=tf.keras.losses.categorical_crossentropy,\n",
    "             metrics=[tf.keras.metrics.categorical_accuracy])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2输入Numpy数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 1000 samples, validate on 200 samples\n",
      "Epoch 1/10\n",
      "1000/1000 [==============================] - 0s 302us/sample - loss: 11.8430 - categorical_accuracy: 0.0940 - val_loss: 11.8829 - val_categorical_accuracy: 0.0850\n",
      "Epoch 2/10\n",
      "1000/1000 [==============================] - 0s 13us/sample - loss: 12.2077 - categorical_accuracy: 0.0850 - val_loss: 12.6407 - val_categorical_accuracy: 0.0850\n",
      "Epoch 3/10\n",
      "1000/1000 [==============================] - 0s 25us/sample - loss: 13.3500 - categorical_accuracy: 0.0940 - val_loss: 14.4725 - val_categorical_accuracy: 0.0850\n",
      "Epoch 4/10\n",
      "1000/1000 [==============================] - 0s 22us/sample - loss: 15.7563 - categorical_accuracy: 0.1070 - val_loss: 17.8371 - val_categorical_accuracy: 0.0850\n",
      "Epoch 5/10\n",
      "1000/1000 [==============================] - 0s 22us/sample - loss: 19.9573 - categorical_accuracy: 0.1000 - val_loss: 23.3407 - val_categorical_accuracy: 0.0750\n",
      "Epoch 6/10\n",
      "1000/1000 [==============================] - 0s 29us/sample - loss: 25.7599 - categorical_accuracy: 0.1010 - val_loss: 29.8918 - val_categorical_accuracy: 0.0750\n",
      "Epoch 7/10\n",
      "1000/1000 [==============================] - 0s 25us/sample - loss: 33.3439 - categorical_accuracy: 0.1030 - val_loss: 39.8591 - val_categorical_accuracy: 0.0650\n",
      "Epoch 8/10\n",
      "1000/1000 [==============================] - 0s 30us/sample - loss: 44.7319 - categorical_accuracy: 0.1010 - val_loss: 53.5336 - val_categorical_accuracy: 0.0550\n",
      "Epoch 9/10\n",
      "1000/1000 [==============================] - 0s 22us/sample - loss: 58.9997 - categorical_accuracy: 0.1050 - val_loss: 69.6580 - val_categorical_accuracy: 0.0500\n",
      "Epoch 10/10\n",
      "1000/1000 [==============================] - 0s 32us/sample - loss: 76.0207 - categorical_accuracy: 0.1040 - val_loss: 89.1147 - val_categorical_accuracy: 0.0700\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f99c0370390>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "train_x = np.random.random((1000, 72))\n",
    "train_y = np.random.random((1000, 10))\n",
    "\n",
    "val_x = np.random.random((200, 72))\n",
    "val_y = np.random.random((200, 10))\n",
    "\n",
    "model.fit(train_x, train_y, epochs=10, batch_size=100,\n",
    "          validation_data=(val_x, val_y))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3tf.data输入数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: Logging before flag parsing goes to stderr.\n",
      "W0308 22:45:22.538554 140299441469184 training_utils.py:1353] Expected a shuffled dataset but input dataset `x` is not shuffled. Please invoke `shuffle()` on input dataset.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "30/30 [==============================] - 0s 4ms/step - loss: 120.5764 - categorical_accuracy: 0.1094 - val_loss: 167.2411 - val_categorical_accuracy: 0.0833\n",
      "Epoch 2/10\n",
      "30/30 [==============================] - 0s 3ms/step - loss: 208.8775 - categorical_accuracy: 0.0983 - val_loss: 272.6065 - val_categorical_accuracy: 0.1250\n",
      "Epoch 3/10\n",
      "30/30 [==============================] - 0s 3ms/step - loss: 323.3857 - categorical_accuracy: 0.0962 - val_loss: 401.4161 - val_categorical_accuracy: 0.0833\n",
      "Epoch 4/10\n",
      "30/30 [==============================] - 0s 3ms/step - loss: 457.8510 - categorical_accuracy: 0.1015 - val_loss: 546.6275 - val_categorical_accuracy: 0.0833\n",
      "Epoch 5/10\n",
      "30/30 [==============================] - 0s 2ms/step - loss: 609.1648 - categorical_accuracy: 0.1197 - val_loss: 712.8060 - val_categorical_accuracy: 0.0938\n",
      "Epoch 6/10\n",
      "30/30 [==============================] - 0s 2ms/step - loss: 780.5939 - categorical_accuracy: 0.0972 - val_loss: 891.2927 - val_categorical_accuracy: 0.0938\n",
      "Epoch 7/10\n",
      "30/30 [==============================] - 0s 3ms/step - loss: 956.0356 - categorical_accuracy: 0.0962 - val_loss: 1068.0684 - val_categorical_accuracy: 0.1354\n",
      "Epoch 8/10\n",
      "30/30 [==============================] - 0s 3ms/step - loss: 1116.4347 - categorical_accuracy: 0.0940 - val_loss: 1220.6263 - val_categorical_accuracy: 0.1042\n",
      "Epoch 9/10\n",
      "30/30 [==============================] - 0s 2ms/step - loss: 1252.2060 - categorical_accuracy: 0.0833 - val_loss: 1364.1936 - val_categorical_accuracy: 0.0833\n",
      "Epoch 10/10\n",
      "30/30 [==============================] - 0s 2ms/step - loss: 1392.8480 - categorical_accuracy: 0.0897 - val_loss: 1507.7679 - val_categorical_accuracy: 0.0938\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f99c47661d0>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset = tf.data.Dataset.from_tensor_slices((train_x, train_y))\n",
    "dataset = dataset.batch(32)\n",
    "dataset = dataset.repeat()\n",
    "val_dataset = tf.data.Dataset.from_tensor_slices((val_x, val_y))\n",
    "val_dataset = val_dataset.batch(32)\n",
    "val_dataset = val_dataset.repeat()\n",
    "\n",
    "model.fit(dataset, epochs=10, steps_per_epoch=30,\n",
    "          validation_data=val_dataset, validation_steps=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.4评估与预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1000/1000 [==============================] - 0s 37us/sample - loss: 1436.8957 - categorical_accuracy: 0.0930\n",
      "30/30 [==============================] - 0s 2ms/step - loss: 1434.2727 - categorical_accuracy: 0.0917\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[1434.2726928710938, 0.09166667]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_x = np.random.random((1000, 72))\n",
    "test_y = np.random.random((1000, 10))\n",
    "model.evaluate(test_x, test_y, batch_size=32)\n",
    "test_data = tf.data.Dataset.from_tensor_slices((test_x, test_y))\n",
    "test_data = test_data.batch(32).repeat()\n",
    "model.evaluate(test_data, steps=30)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.03482518 0.         0.         ... 0.         0.01383898 0.00980373]\n",
      " [0.04969706 0.         0.         ... 0.         0.01390861 0.01697475]\n",
      " [0.05528085 0.         0.         ... 0.         0.01484961 0.01221495]\n",
      " ...\n",
      " [0.02390553 0.         0.         ... 0.         0.0051297  0.01121524]\n",
      " [0.04970536 0.         0.         ... 0.         0.00756324 0.01169133]\n",
      " [0.0402186  0.         0.         ... 0.         0.00886078 0.01124491]]\n"
     ]
    }
   ],
   "source": [
    "# predict\n",
    "result = model.predict(test_x, batch_size=32)\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.构建高级模型\n",
    "\n",
    "### 4.1函数式api\n",
    "tf.keras.Sequential 模型是层的简单堆叠，无法表示任意模型。使用 Keras 函数式 API 可以构建复杂的模型拓扑，例如：\n",
    "\n",
    "多输入模型，\n",
    "\n",
    "多输出模型，\n",
    "\n",
    "具有共享层的模型（同一层被调用多次），\n",
    "\n",
    "具有非序列数据流的模型（例如，残差连接）。\n",
    "\n",
    "**使用函数式 API 构建的模型具有以下特征：**\n",
    "\n",
    "层实例可调用并返回张量。\n",
    "输入张量和输出张量用于定义 tf.keras.Model 实例。\n",
    "此模型的训练方式和 Sequential 模型一样。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/5\n",
      "1000/1000 [==============================] - 0s 125us/sample - loss: 12.9786 - accuracy: 0.0980\n",
      "Epoch 2/5\n",
      "1000/1000 [==============================] - 0s 58us/sample - loss: 19.5004 - accuracy: 0.1060\n",
      "Epoch 3/5\n",
      "1000/1000 [==============================] - 0s 44us/sample - loss: 34.5295 - accuracy: 0.1040\n",
      "Epoch 4/5\n",
      "1000/1000 [==============================] - 0s 60us/sample - loss: 65.7156 - accuracy: 0.0950\n",
      "Epoch 5/5\n",
      "1000/1000 [==============================] - 0s 51us/sample - loss: 118.1614 - accuracy: 0.1190\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f998c243d68>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input_x = tf.keras.Input(shape=(72,))\n",
    "hidden1 = layers.Dense(32, activation='relu')(input_x)\n",
    "hidden2 = layers.Dense(16, activation='relu')(hidden1)\n",
    "pred = layers.Dense(10, activation='softmax')(hidden2)\n",
    "\n",
    "model = tf.keras.Model(inputs=input_x, outputs=pred)\n",
    "model.compile(optimizer=tf.keras.optimizers.Adam(0.001),\n",
    "             loss=tf.keras.losses.categorical_crossentropy,\n",
    "             metrics=['accuracy'])\n",
    "model.fit(train_x, train_y, batch_size=32, epochs=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.2模型子类化\n",
    "通过对 tf.keras.Model 进行子类化并定义您自己的前向传播来构建完全可自定义的模型。在 __init__ 方法中创建层并将它们设置为类实例的属性。在 call 方法中定义前向传播"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/5\n",
      "1000/1000 [==============================] - 0s 159us/sample - loss: 14.8053 - accuracy: 0.0990\n",
      "Epoch 2/5\n",
      "1000/1000 [==============================] - 0s 95us/sample - loss: 22.0737 - accuracy: 0.0960\n",
      "Epoch 3/5\n",
      "1000/1000 [==============================] - 0s 62us/sample - loss: 28.5564 - accuracy: 0.0990\n",
      "Epoch 4/5\n",
      "1000/1000 [==============================] - 0s 84us/sample - loss: 33.0303 - accuracy: 0.1010\n",
      "Epoch 5/5\n",
      "1000/1000 [==============================] - 0s 72us/sample - loss: 36.4832 - accuracy: 0.1040\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f995c46a198>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class MyModel(tf.keras.Model):\n",
    "    def __init__(self, num_classes=10):\n",
    "        super(MyModel, self).__init__(name='my_model')\n",
    "        self.num_classes = num_classes\n",
    "        self.layer1 = layers.Dense(32, activation='relu')\n",
    "        self.layer2 = layers.Dense(num_classes, activation='softmax')\n",
    "    def call(self, inputs):\n",
    "        h1 = self.layer1(inputs)\n",
    "        out = self.layer2(h1)\n",
    "        return out\n",
    "    \n",
    "    def compute_output_shape(self, input_shape):\n",
    "        shape = tf.TensorShape(input_shape).as_list()\n",
    "        shape[-1] = self.num_classes\n",
    "        return tf.TensorShape(shape)\n",
    "\n",
    "model = MyModel(num_classes=10)\n",
    "model.compile(optimizer=tf.keras.optimizers.RMSprop(0.001),\n",
    "             loss=tf.keras.losses.categorical_crossentropy,\n",
    "             metrics=['accuracy'])\n",
    "\n",
    "model.fit(train_x, train_y, batch_size=16, epochs=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.3自定义层\n",
    "通过对 tf.keras.layers.Layer 进行子类化并实现以下方法来创建自定义层：\n",
    "\n",
    "build：创建层的权重。使用 add_weight 方法添加权重。\n",
    "\n",
    "call：定义前向传播。\n",
    "\n",
    "compute_output_shape：指定在给定输入形状的情况下如何计算层的输出形状。\n",
    "或者，可以通过实现 get_config 方法和 from_config 类方法序列化层。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/5\n",
      "1000/1000 [==============================] - 0s 127us/sample - loss: 11.6270 - accuracy: 0.1130\n",
      "Epoch 2/5\n",
      "1000/1000 [==============================] - 0s 73us/sample - loss: 11.6259 - accuracy: 0.1130\n",
      "Epoch 3/5\n",
      "1000/1000 [==============================] - 0s 72us/sample - loss: 11.6243 - accuracy: 0.1080\n",
      "Epoch 4/5\n",
      "1000/1000 [==============================] - 0s 115us/sample - loss: 11.6234 - accuracy: 0.1070\n",
      "Epoch 5/5\n",
      "1000/1000 [==============================] - 0s 85us/sample - loss: 11.6219 - accuracy: 0.1070\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f994a6a07b8>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class MyLayer(layers.Layer):\n",
    "    def __init__(self, output_dim, **kwargs):\n",
    "        self.output_dim = output_dim\n",
    "        super(MyLayer, self).__init__(**kwargs)\n",
    "    \n",
    "    def build(self, input_shape):\n",
    "        shape = tf.TensorShape((input_shape[1], self.output_dim))\n",
    "        self.kernel = self.add_weight(name='kernel1', shape=shape,\n",
    "                                   initializer='uniform', trainable=True)\n",
    "        super(MyLayer, self).build(input_shape)\n",
    "    \n",
    "    def call(self, inputs):\n",
    "        return tf.matmul(inputs, self.kernel)\n",
    "\n",
    "    def compute_output_shape(self, input_shape):\n",
    "        shape = tf.TensorShape(input_shape).as_list()\n",
    "        shape[-1] = self.output_dim\n",
    "        return tf.TensorShape(shape)\n",
    "\n",
    "    def get_config(self):\n",
    "        base_config = super(MyLayer, self).get_config()\n",
    "        base_config['output_dim'] = self.output_dim\n",
    "        return base_config\n",
    "\n",
    "    @classmethod\n",
    "    def from_config(cls, config):\n",
    "        return cls(**config)\n",
    "    \n",
    "model = tf.keras.Sequential(\n",
    "[\n",
    "    MyLayer(10),\n",
    "    layers.Activation('softmax')\n",
    "])\n",
    "\n",
    "\n",
    "model.compile(optimizer=tf.keras.optimizers.RMSprop(0.001),\n",
    "             loss=tf.keras.losses.categorical_crossentropy,\n",
    "             metrics=['accuracy'])\n",
    "\n",
    "model.fit(train_x, train_y, batch_size=16, epochs=5)\n",
    "\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.3回调"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 1000 samples, validate on 200 samples\n",
      "Epoch 1/5\n",
      "1000/1000 [==============================] - 0s 214us/sample - loss: 11.6202 - accuracy: 0.1050 - val_loss: 11.5276 - val_accuracy: 0.0950\n",
      "Epoch 2/5\n",
      "1000/1000 [==============================] - 0s 106us/sample - loss: 11.6219 - accuracy: 0.1080 - val_loss: 11.5256 - val_accuracy: 0.1050\n",
      "Epoch 3/5\n",
      "1000/1000 [==============================] - 0s 77us/sample - loss: 11.6234 - accuracy: 0.1020 - val_loss: 11.5226 - val_accuracy: 0.1100\n",
      "Epoch 4/5\n",
      "1000/1000 [==============================] - 0s 94us/sample - loss: 11.6213 - accuracy: 0.0980 - val_loss: 11.5266 - val_accuracy: 0.1050\n",
      "Epoch 5/5\n",
      "1000/1000 [==============================] - 0s 88us/sample - loss: 11.6211 - accuracy: 0.1010 - val_loss: 11.5253 - val_accuracy: 0.1150\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7f995c4b7e80>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "callbacks = [\n",
    "    tf.keras.callbacks.EarlyStopping(patience=2, monitor='val_loss'),\n",
    "    tf.keras.callbacks.TensorBoard(log_dir='./logs')\n",
    "]\n",
    "model.fit(train_x, train_y, batch_size=16, epochs=5,\n",
    "         callbacks=callbacks, validation_data=(val_x, val_y))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5保持和恢复\n",
    "### 5.1权重保存"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = tf.keras.Sequential([\n",
    "layers.Dense(64, activation='relu'),\n",
    "layers.Dense(10, activation='softmax')])\n",
    "\n",
    "model.compile(optimizer=tf.keras.optimizers.Adam(0.001),\n",
    "              loss='categorical_crossentropy',\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "model.save_weights('./weights/model')\n",
    "model.load_weights('./weights/model')\n",
    "model.save_weights('./model.h5')\n",
    "model.load_weights('./model.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.2保存网络结构"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'backend': 'tensorflow',\n",
      " 'class_name': 'Sequential',\n",
      " 'config': {'layers': [{'class_name': 'Dense',\n",
      "                        'config': {'activation': 'relu',\n",
      "                                   'activity_regularizer': None,\n",
      "                                   'bias_constraint': None,\n",
      "                                   'bias_initializer': {'class_name': 'Zeros',\n",
      "                                                        'config': {}},\n",
      "                                   'bias_regularizer': None,\n",
      "                                   'dtype': None,\n",
      "                                   'kernel_constraint': None,\n",
      "                                   'kernel_initializer': {'class_name': 'GlorotUniform',\n",
      "                                                          'config': {'seed': None}},\n",
      "                                   'kernel_regularizer': None,\n",
      "                                   'name': 'dense_17',\n",
      "                                   'trainable': True,\n",
      "                                   'units': 64,\n",
      "                                   'use_bias': True}},\n",
      "                       {'class_name': 'Dense',\n",
      "                        'config': {'activation': 'softmax',\n",
      "                                   'activity_regularizer': None,\n",
      "                                   'bias_constraint': None,\n",
      "                                   'bias_initializer': {'class_name': 'Zeros',\n",
      "                                                        'config': {}},\n",
      "                                   'bias_regularizer': None,\n",
      "                                   'dtype': None,\n",
      "                                   'kernel_constraint': None,\n",
      "                                   'kernel_initializer': {'class_name': 'GlorotUniform',\n",
      "                                                          'config': {'seed': None}},\n",
      "                                   'kernel_regularizer': None,\n",
      "                                   'name': 'dense_18',\n",
      "                                   'trainable': True,\n",
      "                                   'units': 10,\n",
      "                                   'use_bias': True}}],\n",
      "            'name': 'sequential_3'},\n",
      " 'keras_version': '2.2.4-tf'}\n"
     ]
    }
   ],
   "source": [
    "# 序列化成json\n",
    "import json\n",
    "import pprint\n",
    "json_str = model.to_json()\n",
    "pprint.pprint(json.loads(json_str))\n",
    "fresh_model = tf.keras.models.model_from_json(json_str)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "backend: tensorflow\n",
      "class_name: Sequential\n",
      "config:\n",
      "  layers:\n",
      "  - class_name: Dense\n",
      "    config:\n",
      "      activation: relu\n",
      "      activity_regularizer: null\n",
      "      bias_constraint: null\n",
      "      bias_initializer:\n",
      "        class_name: Zeros\n",
      "        config: {}\n",
      "      bias_regularizer: null\n",
      "      dtype: null\n",
      "      kernel_constraint: null\n",
      "      kernel_initializer:\n",
      "        class_name: GlorotUniform\n",
      "        config: {seed: null}\n",
      "      kernel_regularizer: null\n",
      "      name: dense_17\n",
      "      trainable: true\n",
      "      units: 64\n",
      "      use_bias: true\n",
      "  - class_name: Dense\n",
      "    config:\n",
      "      activation: softmax\n",
      "      activity_regularizer: null\n",
      "      bias_constraint: null\n",
      "      bias_initializer:\n",
      "        class_name: Zeros\n",
      "        config: {}\n",
      "      bias_regularizer: null\n",
      "      dtype: null\n",
      "      kernel_constraint: null\n",
      "      kernel_initializer:\n",
      "        class_name: GlorotUniform\n",
      "        config: {seed: null}\n",
      "      kernel_regularizer: null\n",
      "      name: dense_18\n",
      "      trainable: true\n",
      "      units: 10\n",
      "      use_bias: true\n",
      "  name: sequential_3\n",
      "keras_version: 2.2.4-tf\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 保持为yaml格式  #需要提前安装pyyaml\n",
    "\n",
    "yaml_str = model.to_yaml()\n",
    "print(yaml_str)\n",
    "fresh_model = tf.keras.models.model_from_yaml(yaml_str)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.3保存整个模型\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/5\n",
      "1000/1000 [==============================] - 0s 131us/sample - loss: 11.6308 - accuracy: 0.1040\n",
      "Epoch 2/5\n",
      "1000/1000 [==============================] - 0s 40us/sample - loss: 11.6897 - accuracy: 0.1080\n",
      "Epoch 3/5\n",
      "1000/1000 [==============================] - 0s 55us/sample - loss: 11.7418 - accuracy: 0.1070\n",
      "Epoch 4/5\n",
      "1000/1000 [==============================] - 0s 43us/sample - loss: 11.7983 - accuracy: 0.1060\n",
      "Epoch 5/5\n",
      "1000/1000 [==============================] - 0s 47us/sample - loss: 11.8449 - accuracy: 0.0910\n"
     ]
    }
   ],
   "source": [
    "model = tf.keras.Sequential([\n",
    "  layers.Dense(10, activation='softmax', input_shape=(72,)),\n",
    "  layers.Dense(10, activation='softmax')\n",
    "])\n",
    "model.compile(optimizer='rmsprop',\n",
    "              loss='categorical_crossentropy',\n",
    "              metrics=['accuracy'])\n",
    "model.fit(train_x, train_y, batch_size=32, epochs=5)\n",
    "model.save('all_model.h5')\n",
    "model = tf.keras.models.load_model('all_model.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6.将keras用于Estimator\n",
    "Estimator API 用于针对分布式环境训练模型。它适用于一些行业使用场景，例如用大型数据集进行分布式训练并导出模型以用于生产"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "W0308 22:45:28.520160 140299441469184 estimator.py:1799] Using temporary folder as model directory: /tmp/tmpzg_jzz1o\n"
     ]
    }
   ],
   "source": [
    "model = tf.keras.Sequential([layers.Dense(10,activation='softmax'),\n",
    "                          layers.Dense(10,activation='softmax')])\n",
    "\n",
    "model.compile(optimizer=tf.keras.optimizers.RMSprop(0.001),\n",
    "              loss='categorical_crossentropy',\n",
    "              metrics=['accuracy'])\n",
    "\n",
    "estimator = tf.keras.estimator.model_to_estimator(model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
